The present disclosure relates generally to field of database search and storage; and more specifically to a hybrid table implementation by using buffer pool as permanent in-memory storage for memory resident-data.
Many large databases are primarily stored on a disk. These disk based databases, often use a buffer to retrieve newly accessed data to increase performance. Disk-based databases often share buffer space to optimize space and increase performance of the system. However, the pooling of the buffer creates performance bottlenecks when sending or receiving data, which are often caused by the disk input-output (I/O) requirements.
To reduce such bottlenecks and avoid the performance issues, in-memory databases are sometimes used. In an in-memory database, the primary location of data is in the physical or permanent memory. Most in-memory databases are characterized by memory optimized data structures and access methods. Performance is greatly enhanced by sorting, storing and retrieving specified data entirely from the in-memory, rather than going to disk based. Using an in-memory database allows access requests to be executed without dealing with the issues of disk I/O bottlenecks provided in the code path. However, using an in-memory database scheme has many trade-offs as the storage cost per data unit is higher for the in-memory database than for a disk memory scheme. This is because a less costly hard disk can be substituted for more memory. In addition, random access memories (RAM) chips used for in-memory database schemes cannot approach the density of a small hard drive. Another issue with in-memory databases is that in some applications, mining on large databases cannot be held into in-memory storage due to space constraints. One solution is to use a hybrid search index of databases which uses both “d” and “m” types of databases.